feat(skills): add episodic problem-framing job to the Data Workstream Coach - #2748

Description

The Data Workstream Coach could route a data scientist to feasibility, experiments, pipelines, and evaluation, but nothing owned the step before all of them: deciding what kind of data-science problem a stated business outcome actually is. A practitioner arriving with "we want to reduce churn" had no job to select. This PR closes that gap by extending capability that already exists rather than adding an eighth skill.

experiment-design now owns a second, clearly separated procedure. Problem-class framing takes a stated business outcome, applies a per-paradigm entry test to nine candidate problem classes, orders the unresolved questions by how many candidates each answer eliminates, and returns the surviving candidates with the reasoning that kept or discarded each one. It never selects, ranks, scores, or confidence-thresholds a candidate, because the data scientist is the subject-matter expert making that call.

Source treatment is deliberately layered and visible in the output. Machine-learning entry tests are paraphrased and reorganized from two Google for Developers pages licensed CC BY 4.0, with both URLs cited and the changes stated in THIRD-PARTY-NOTICES. The NEOS Guide reserves rights, so it is cited as further reading only and the procedure states no optimization classification of its own. Everything else, including cross-paradigm routing and the broader "is this even a modelling problem" judgement, is repository-original. Every claim carries a [G], [O], or [A] label so a practitioner can see which is which.

A second, unrelated commit corrects a licensing accuracy problem found while writing the attribution above. Five skills declared license: mixed, which is not an SPDX identifier and tells a consumer nothing.

• feat(skills): added references/problem-framing.md to experiment-design with a source block, provenance labels, a single pre-analysis stop, a six-step procedure, nine candidate problem classes with entry tests, a coordinator-ready gap contract, an output contract, and an explicit never-does list.
• feat(skills): broadened the experiment-design contract so the Goal, Inputs, reference index, success criteria, constraints, ownership boundaries, stop rules, and attribution all admit problem-class framing while leaving the experiment workflow untouched. A confirmed framing request reads one file; experiment requests continue to read mve-coaching.md or experiment-readiness.md.
• feat(skills): registered problem-framing in the Data Workstream job registry as episodic, routing to experiment-design with no supporting route, disambiguated it from the existing experiment job, and recorded two completion shapes so an unstated outcome completes as real work rather than discarded-cleanly.
  • The session-state schema gained the matching problem-framing job slot. The state protocol requires the jobs key set to equal the registry identifiers and stops on mismatch, so registering the job without its slot would have halted every Data Workstream session rather than just framing.
• feat(agents): updated both hardcoded experiment-design capability statements in the Data Workstream Coach so a problem-framing route announcement is accurate. No framing procedure, entry test, or job enumeration was imported into the agent; it stays a thin coordinator.
• feat(evals): added one problem-framing-job-contract expectation and one comparative two-case stimulus. Case A withholds the business outcome and must stop with an assigned gap; Case B supplies an outcome but leaves a high-power question unresolved and must elicit, mark unknown, continue, and select nothing. Nine independent graders cover both branches, including a narrow negative grader that fails only when the response turns the missing outcome into an experiment subject.
• feat(docs): recorded the Google CC BY 4.0 sources in THIRD-PARTY-NOTICES with the license URI, both source URLs, the exact usage scope, and a statement of changes.
• fix(skills): replaced license: mixed on five skills with compound SPDX expressions and gave each a per-file license allocation table.
  • vex is Apache-2.0 AND CC-BY-4.0, privacy-standards is CC-BY-4.0 AND CC-BY-SA-3.0, functional-planner is CC-BY-4.0 AND CC-BY-SA-4.0, c4-architecture is CC-BY-4.0 AND MIT, and rai-standards resolves to plain CC-BY-4.0 because its sources are public-domain or paraphrase-only and add no term.
• fix(instructions): rewrote the mixed-content section of the licensing posture so it prescribes the compound expression plus allocation table, explains that package-level AND is the redistributor conjunction rather than a per-file claim, and removed the contradictory checklist bullet that still required license: mixed.

Related Issue(s)

Closes #2747
Related to #1543

Type of Change

Select all that apply:

Code & Documentation:

• Bug fix (non-breaking change fixing an issue)
• New feature (non-breaking change adding functionality)
• Breaking change (fix or feature causing existing functionality to change)
• Documentation update

Infrastructure & Configuration:

• GitHub Actions workflow
• Linting configuration (markdown, PowerShell, etc.)
• Security configuration
• DevContainer configuration
• Dependency update

AI Artifacts:

• Reviewed contribution with hve-builder and addressed all actionable findings
• Copilot instructions (.github/instructions/*.instructions.md)
• Copilot prompt (.github/prompts/*.prompt.md)
• Copilot agent (.github/agents/*.agent.md)
• Copilot skill (.github/skills/*/SKILL.md)
• Copilot hook (.github/hooks/*/*.json)
• Eval spec added/updated for changed AI artifacts (evals/)

Note for AI Artifact Contributors:

• Agents: Research, indexing/referencing other project (using standard VS Code GitHub Copilot/MCP tools), planning, and general implementation agents likely already exist. Review .github/agents/ before creating new ones.
• Skills: Must include both bash and PowerShell scripts. See Skills.
• Model Versions: Contributions MUST target models listed in the model catalog (scripts/linting/model-catalog.json) whose provider appears in providerAllowlist and whose status is ga or preview. Run npm run lint:models to validate references.
• See Agents Not Accepted and Model Version Requirements.

Other:

• Script/automation (.ps1, .sh, .py)
• Other (please describe):

Sample Prompts (for AI Artifact Contributions)

User Request:

Start a Data Workstream Coach session, then select the new job when it is offered:

I confirm the problem-framing job. We want to reduce churn in our subscription base, and I need to know what kind of data-science problem that actually is before we commit to anything.

Execution Flow:

1. The coach loads data-workstream-foundation and presents the job list from references/job-registry.md. problem-framing appears as an episodic job routing to experiment-design. Nothing activates until the user confirms.
2. On confirmation, the coach records the job in session state and loads experiment-design. Its reference index sends a confirmed framing request to references/problem-framing.md and to nothing else, so experiment vetting criteria and MVE fallbacks never load.
3. The procedure checks for a stated business outcome. If it is missing, it coaches for it with one decision-bearing question, and if the user declines, it completes with a single outcome-definition gap and an explicit statement that no candidate analysis was performed.
4. With an outcome stated, it applies the entry test for all nine candidate problem classes, keeping any class whose test passes or cannot yet be evaluated and recording the failure reason for any it discards.
5. It orders the unresolved questions by discriminating power, asks the user for each value, and marks a value unknown only after elicitation fails. An unanswered question never terminates the run.
6. It returns the output contract and hands control back to the coach, which asks before starting any next job.

Output Artifacts:

A chat-rendered framing result. The framing procedure itself writes no durable file. The shape is:

Restated outcome [O]: Reduce voluntary churn among monthly subscribers.

Surviving candidates

| Class                     | Entry test result                                   | Label |
|---------------------------|-----------------------------------------------------|-------|
| Supervised classification | Churn is labelled historically; features exist      | [G]   |
| Time-to-event analysis    | "When will they churn" is still open                | [O]   |
| Causal inference          | Only if the question is what a retention offer does | [O]   |

Discarded

| Forecasting | The forecast would change the outcome it predicts | [O] |

Discriminating questions

1. Is the decision "who to target" or "what intervention works"? -- unknown
2. Is a per-customer timing estimate required? -- unknown

Gaps

| Needed fact                | Why it matters               | Answer holder | Coordinator | Status  |
|----------------------------|------------------------------|---------------|-------------|---------|
| Decision the model informs | Splits classification/causal | unassigned    | unassigned  | unknown |

Success Indicators:

• The response names experiment-design as the owner and the invocation as episodic.
• Every candidate carries a [G], [O], or [A] label, and no claim implies that an external standard adjudicates across paradigms.
• No candidate is selected, ranked, scored, or confidence-thresholded, and no next job starts without a separate confirmation.
• A withheld business outcome produces an assigned gap rather than an MVE or an experiment.
• Optimization structure is never classified; the response points to the cited NEOS pages instead.

Testing

Automated validation run on this branch after rebasing onto origin/main. Each command was run independently so a late failure could not hide behind a chained exit.

Diff-based and manual assessments performed:

• Registry-to-session-state parity was checked directly: both job key sets are identical at ten jobs. A simulated resume against a state record missing the slot correctly stopped and reported rather than silently repairing or dropping the job.
• All nine grader regexes were compiled and confirmed valid.
• An independent static review of the five changed artifacts returned Revise with two Critical findings and one High finding. All three were fixed: a cite-only source boundary breach in the optimization section, the missing session-state slot, and a negative grader that would have failed a compliant response for naming the boundary it was observing.
• An independent behavior test ran seven black-box scenarios in read-only simulation and established routing, the outcome stop, gap continuation with unassigned ownership, refusal to classify optimization structure, refusal to rank or recommend, and refusal to auto-advance into feasibility or experiment work.
• Native runtime behavior and the durable-write scan gate were not exercised. The behavior evidence is simulation, not a live coach invocation.

Checklist

Required Checks

• Documentation is updated (if applicable)
• Files follow existing naming conventions
• Changes are backwards compatible (if applicable)
• Tests added for new functionality (if applicable)

AI Artifact Contributions

• Used hve-builder review mode to review contribution
• Addressed all actionable findings from the hve-builder review
• Verified contribution follows common standards and type-specific requirements

Required Local Checks

The following local-safe validation commands must pass before merging:

• Local validation aggregate: npm run validate:local
• Documentation validation (if docs changed): npm run validate:docs
• Spell checking: npm run spell-check
• Link validation: npm run lint:md-links

Security Considerations

• This PR does not contain any sensitive or NDA information
• Any new dependencies have been reviewed for security issues (N/A — no dependency or lockfile changes)
• Security-related scripts follow the principle of least privilege (N/A — no files under scripts/security/ changed)

Additional Notes

On the licensing commit. It is genuinely unrelated to the framing capability and is kept as a separate commit for that reason. It was found while writing the attribution for the new Google sources: license: mixed is not an SPDX identifier, and the frontmatter schema documents that field as one. The schema currently accepts any 1–128 character string, which is how the placeholder spread to five skills. Tightening that schema to an enum is worth a follow-up but is not attempted here.

On scope discipline. The capability was deliberately built into an existing skill. No eighth Data Workstream skill was added, the seven-skill boundary contract and its expectation are untouched, plugin.json is unchanged, and no new instruction file, schema, validator, dependency, or feasibility-profile version was introduced.

Known residual items.

• Four graders in the new stimulus are not literally case-scoped, and two Case A patterns use an unbounded lazy match that could traverse into Case B. This is a false-pass risk only, not a false-failure risk, and is worth tightening in a follow-up.
• The Seven-skill boundaries capability column in the job registry still describes experiment-design in experiment-only terms. Editing that table would touch the seven-skill-boundaries expectation, so it was intentionally left for the existing count-free-naming follow-up.
• THIRD-PARTY-NOTICES carries no retrieval dates for any entry, which matches its existing convention. Retrieval dates for the new sources are recorded in the research evidence instead.

🧭 - Generated by Copilot